Linking Names and Faces: Seeing the Problem in Different Ways

In this paper we report on our experiments on linking names and faces as found in images and captions of online news websites. Whereas previously, the focus has been mostly on assigning names to the faces, we generalize this framework, exploiting the (a)symmetry between the visual and textual modalities. This leads to different schemes for assigning names to faces, assigning faces to names, and establishing name-face link pairs. On top of that, we investigate the use of textual and visual structural information to predict the presence of the corresponding entity in the other modality. This further improves the accuracy of the cross-media linking results.